When a computer image, which may be a two-dimensional (2D) or three-dimensional (3D) image, is generated from a real-world scene, the computer image may include a semi-reflective surface (e.g. window) that creates both a reflection layer (e.g. reflection radiance map) and a transmission layer (e.g. transmission radiance map). The reflection layer may include a reflection of a scene in front of the semi-reflective surface from the perspective of a camera capturing the real-world scene, where the reflected scene may or may not be behind the camera. On the other hand, the transmission layer may include transmission of a scene located behind the semi-reflective surface from the perspective of the camera, where the transmission scene is visible through the semi-reflective surface. Similar to a person viewing the real-world scene from different locations, angles, etc., the reflection and transmission layers may change, and also move relative to each other, as the viewpoint of the camera changes.
Unfortunately, the performance of many computer applications that process scenes having semi-reflective surfaces is negatively impacted as a result of the reflections caused by the semi-reflective surfaces. For example, some applications including computer vision applications, such as multi-view stereo (MVS), image registration, or simultaneous localization and mapping (SLAM), depend on the ability to find corresponding image patches across images taken from different views of a particular scene. However, this ability traditionally relies on the assumption that a pixel captures radiance from a single object, such that pixels with a same radiance across the images (even after accounting for geometric distortion) are correlated. However, semi-reflective surfaces break this assumption by creating a superposition of two images (the images of the reflected and transmitted radiance) at the same pixel, which may cause a variance in the radiance of the pixels that would otherwise be assumed to correlate across the images. To hold the assumption true, there is a need to separate reflection and transmission images included on a semi-reflective surface of a computer generated image, so that the image processing applications can process the reflection and/or transmission images independently.
Some prior methods have attempted to provide this separation of reflection and transmission images by capturing multiple polarization images (i.e. images captured at different polarization angles). Since these images offer independent measurements of the same scene, the reflection and transmission have simply been separated using independent component analysis of the captured images. However, these prior methods make strong assumptions about the different images, such as that the viewing angle is roughly the Brewster angle where polarization helps the most, that the reflection will be blurry, that the semi-reflective surface will be flat, etc. Since these assumptions do not necessarily hold true in the real-world, the quality of the results of these simplified methods degrades significantly when applied to real-world images versus synthetic images.
There is a need for addressing these issues and/or other issues associated with the prior art.